narrative bias
A Close Reading Approach to Gender Narrative Biases in AI-Generated Stories
Raffini, Daniel, Macori, Agnese, Angelini, Marco, Catarci, Tiziana
--The paper explores the study of gender-based narrative biases in stories generated by ChatGPT, Gemini, and Claude. The prompt design draws on Propp's character classifications and Freytag's narrative structure. The stories are analyzed through a close reading approach, with particular attention to adherence to the prompt, gender distribution of characters, physical and psychological descriptions, actions, and finally, plot development and character relationships. The results reveal the persistence of biases -- especially implicit ones -- in the generated stories and highlight the importance of assessing biases at multiple levels using an interpretative approach. In recent years, considerable attention has been paid to addressing the problem of bias in Large Language Models (LLMs). Despite ongoing efforts and improvements, LLMs still often do not adequately represent diversity and continue to propagate various forms of societal bias in their output [1] [2] [3]. The extensive use of LLMs for content creation and text generation makes this issue increasingly urgent. Regarding gender bias, studies have explored different aspects, such as the correlation between gender and occupation [4] [5], personas [6] [7], or the use of adjectives [8]. Many of these studies also compared LLMs' correlations with official social data on occupation and human perceptions [5] [9]. Methodologies for studying bias in LLMs can be divided into intrinsic and extrinsic approaches [10] [11]. The intrinsic approach includes embedding-and probability-based bias, while the extrinsic approach focuses on generation-based bias [12]. A recent study from UNESCO [13] provides a comprehensive application of various approaches by studying the connection of gendered words, asking LLMs to complete sentences, and generating entire stories. There are different modes of gender bias and stereotype propagation, and it is important to evaluate the issue from various points of view.
- Europe > Italy > Lazio > Rome (0.04)
- Europe > France > Île-de-France > Paris > Paris (0.04)
- North America > United States > Texas > Travis County > Austin (0.04)
- (4 more...)
Machine Guides, Human Supervises: Interactive Learning with Global Explanations
Popordanoska, Teodora, Kumar, Mohit, Teso, Stefano
We introduce explanatory guided learning (XGL), a novel interactive learning strategy in which a machine guides a human supervisor toward selecting informative examples for a classifier. The guidance is provided by means of global explanations, which summarize the classifier's behavior on different regions of the instance space and expose its flaws. Compared to other explanatory interactive learning strategies, which are machine-initiated and rely on local explanations, XGL is designed to be robust against cases in which the explanations supplied by the machine oversell the classifier's quality. Moreover, XGL leverages global explanations to open up the black-box of human-initiated interaction, enabling supervisors to select informative examples that challenge the learned model. By drawing a link to interactive machine teaching, we show theoretically that global explanations are a viable approach for guiding supervisors. Our simulations show that explanatory guided learning avoids overselling the model's quality and performs comparably or better than machine- and human-initiated interactive learning strategies in terms of model quality.
Toward Machine-Guided, Human-Initiated Explanatory Interactive Learning
Recent work has demonstrated the promise of combining local explanations with active learning for understanding and supervising black-box models. Here we show that, under specific conditions, these algorithms may misrepresent the quality of the model being learned. The reason is that the machine illustrates its beliefs by predicting and explaining the labels of the query instances: if the machine is unaware of its own mistakes, it may end up choosing queries on which it performs artificially well. This biases the "narrative" presented by the machine to the user.We address this narrative bias by introducing explanatory guided learning, a novel interactive learning strategy in which: i) the supervisor is in charge of choosing the query instances, while ii) the machine uses global explanations to illustrate its overall behavior and to guide the supervisor toward choosing challenging, informative instances. This strategy retains the key advantages of explanatory interaction while avoiding narrative bias and compares favorably to active learning in terms of sample complexity.
Toward Machine-Guided, Human-Initiated Explanatory Interactive Learning
Popordanoska, Teodora, Kumar, Mohit, Teso, Stefano
Recent work has demonstrated the promise of combining local explanations with active learning for understanding and supervising black-box models. Here we show that, under specific conditions, these algorithms may misrepresent the quality of the model being learned. The reason is that the machine illustrates its beliefs by predicting and explaining the labels of the query instances: if the machine is unaware of its own mistakes, it may end up choosing queries on which it performs artificially well. This biases the "narrative" presented by the machine to the user. We address this narrative bias by introducing explanatory guided learning, a novel interactive learning strategy in which: i) the supervisor is in charge of choosing the query instances, while ii) the machine uses global explanations to illustrate its overall behavior and to guide the supervisor toward choosing challenging, informative instances. This strategy retains the key advantages of explanatory interaction while avoiding narrative bias and compares favorably to active learning in terms of sample complexity. An initial empirical evaluation with a clustering-based prototype highlights the promise of our approach.
- Europe > Italy > Trentino-Alto Adige/Südtirol > Trentino Province > Trento (0.04)
- Europe > Belgium > Flanders > Flemish Brabant > Leuven (0.04)